Tutorial 5: Hypothesis Testing

It is often the case that we want to infer information using collected data, such as whether two samples can be considered to be from the same population, whether one sample has systematically larger values than another, or whether samples can be considered to be correlated. Such hypotheses may be formally tested using inferential statistics, allowing conclusions to be drawn, allowing the potential for objective decision making in the presence of a stochastic element. The general idea is to predict the likelihood of an event associated with a given statement (i.e. the hypothesis) occurring by chance, given the observed data and available information. If it is determined that the event is highly unlikely to randomly occur, then the hypothesis may be rejected, concluding that it is unlikely for the hypothesis to be correct. Conversely, if it is determined that there is a reasonable chance that the event may randomly occur, then it is concluded that it is not possible to prove nor disprove the hypothesis, using the particular test performed, given the observed data and available information. Conceptually, this is similar to saying that the hypothesis is ‘innocent until proven guilty’. Such hypothesis testing is at the core of applied statistics and data analysis. The ability to draw valid conclusions from such testing is subject to certain assumptions, the most simple/universal of which being the base assumptions that the observed data are ordinal, and are typical of the populations they represent. However, assumptions are often also made about the underlying distribution of the data. Different statistical tests require different assumptions to be satisfied in order to be validly used. Tests that make fewer assumptions about the nature of the data are inherently applicable to wider classes of problems, whilst often suffering from reduced Statistical Power (i.e. reduced ability to correctly detect thus reject the hypothesis in cases when the hypothesis is truly incorrect).